Participatory development of decision support systems: which features of the process lead to improved uptake and better outcomes?

Decision support systems (DSSs) are important in decision-making environments with conflicting interests. Many DSSs developed have not been used in practice. Experts argue that these tools do not respond to real user needs and that the inclusion of stakeholders in the development process is the solution. However, it is not clear which features of participatory development of DSSs result in improved uptake and better outcomes. A review of papers, reporting on case studies where DSSs and other decision tools (information systems, software and scenario tools) were developed with elements of participation, was carried out. The cases were analysed according to a framework created as part of this research; it includes criteria to evaluate the development process and the outcomes. Relevant aspects to consider in the participatory development processes include establishing clear objectives, timing and location of the process; keeping discussions on track; favouring participation and interaction of individuals and groups; and challenging creative thinking of the tool and future scenarios. The case studies that address these issues show better outcomes; however, there is a large degree of uncertainty concerning them because developers have typically neither asked participants about their perceptions of the processes and resultant tools nor have they monitored the use and legacy of the tools over the long term.The authors would like to thank COST Action FP0804-Forest Management Decision Support Systems (FORSYS) for financing a three month Short-Term Scientific Mission (STSM) in Forest Research (Roslin, UK) in 2012, making possible this research; Spanish Ministry of Economy and Competitiveness for supporting the project Multicriteria Techniques and Participatory Decision-Making for Sustainable Management (Ref. ECO2011-27369) where the leading author is involved; and the Regional Ministry of Education, Culture and Sports (Valencia, Spain) for financing a research fellowship (Ref. ACIF/2010/248).Valls Donderis, P.; Ray, D.; Peace, A.; Stewart, A.; Lawrence, A.; Galiana, F. (2013). Participatory development of decision support systems: which features of the process lead to improved uptake and better outcomes?. Scandinavian Journal of Forest Research. 29(1):71-83. https://doi.org/10.1080/02827581.2013.837950S7183291Arnstein, S. R. (1969). A Ladder Of Citizen Participation. Journal of the American Institute of Planners, 35(4), 216-224. doi:10.1080/01944366908977225Atwell, R. C., Schulte, L. A., & Westphal, L. M. (2011). Tweak, Adapt, or Transform: Policy Scenarios in Response to Emerging Bioenergy Markets in the U.S. Corn Belt. Ecology and Society, 16(1). doi:10.5751/es-03854-160110Barac, A., Kellner, K., & De Klerk, N. (2004). Land User Participation in Developing a Computerised Decision Support System for Combating Desertification. Environmental Monitoring and Assessment, 99(1-3), 223-231. doi:10.1007/s10661-004-4022-6Bennet, A., & Bennet, D. (2008). The Decision-Making Process in a Complex Situation. Handbook on Decision Support Systems 1, 3-20. doi:10.1007/978-3-540-48713-5_1Blackstock, K. L., Kelly, G. J., & Horsey, B. L. (2007). Developing and applying a framework to evaluate participatory research for sustainability. Ecological Economics, 60(4), 726-742. doi:10.1016/j.ecolecon.2006.05.014Breuer, N. E., Cabrera, V. E., Ingram, K. T., Broad, K., & Hildebrand, P. E. (2007). AgClimate: a case study in participatory decision support system development. Climatic Change, 87(3-4), 385-403. doi:10.1007/s10584-007-9323-7Bunch, M. J., & Dudycha, D. J. (2004). Linking conceptual and simulation models of the Cooum River: collaborative development of a GIS-based DSS for environmental management. Computers, Environment and Urban Systems, 28(3), 247-264. doi:10.1016/s0198-9715(03)00021-8Byrne, E., & Sahay, S. (2007). Participatory design for social development: A South African case study on community-based health information systems. Information Technology for Development, 13(1), 71-94. doi:10.1002/itdj.20052Cain, J. ., Jinapala, K., Makin, I. ., Somaratna, P. ., Ariyaratna, B. ., & Perera, L. . (2003). Participatory decision support for agricultural management. A case study from Sri Lanka. Agricultural Systems, 76(2), 457-482. doi:10.1016/s0308-521x(02)00006-9Chakraborty, A. (2011). Enhancing the role of participatory scenario planning processes: Lessons from Reality Check exercises. Futures, 43(4), 387-399. doi:10.1016/j.futures.2011.01.004Cinderby, S., Bruin, A. de, Mbilinyi, B., Kongo, V., & Barron, J. (2011). Participatory geographic information systems for agricultural water management scenario development: A Tanzanian case study. Physics and Chemistry of the Earth, Parts A/B/C, 36(14-15), 1093-1102. doi:10.1016/j.pce.2011.07.039Drew, C. H., Nyerges, T. L., & Leschine, T. M. (2004). Promoting Transparency of Long‐Term Environmental Decisions: The Hanford Decision Mapping System Pilot Project. Risk Analysis, 24(6), 1641-1664. doi:10.1111/j.0272-4332.2004.00556.xDriedger, S. M., Kothari, A., Morrison, J., Sawada, M., Crighton, E. J., & Graham, I. D. (2007). Using participatory design to develop (public) health decision support systems through GIS. International Journal of Health Geographics, 6(1), 53. doi:10.1186/1476-072x-6-53Evers, M. (2008). An analysis of the requirements for DSS on integrated river basin management. Management of Environmental Quality: An International Journal, 19(1), 37-53. doi:10.1108/14777830810840354Iivari, N. (2011). Participatory design in OSS development: interpretive case studies in company and community OSS development contexts. Behaviour & Information Technology, 30(3), 309-323. doi:10.1080/0144929x.2010.503351Innes, J. E., & Booher, D. E. (1999). Consensus Building and Complex Adaptive Systems. Journal of the American Planning Association, 65(4), 412-423. doi:10.1080/01944369908976071Jakku, E., & Thorburn, P. J. (2010). A conceptual framework for guiding the participatory development of agricultural decision support systems. Agricultural Systems, 103(9), 675-682. doi:10.1016/j.agsy.2010.08.007Jessel, B., & Jacobs, J. (2005). Land use scenario development and stakeholder involvement as tools for watershed management within the Havel River Basin. Limnologica, 35(3), 220-233. doi:10.1016/j.limno.2005.06.006Kautz, K. (2011). Investigating the design process: participatory design in agile software development. Information Technology & People, 24(3), 217-235. doi:10.1108/09593841111158356Kowalski, K., Stagl, S., Madlener, R., & Omann, I. (2009). Sustainable energy futures: Methodological challenges in combining scenarios and participatory multi-criteria analysis. European Journal of Operational Research, 197(3), 1063-1074. doi:10.1016/j.ejor.2007.12.049Lawrence, A. (2006). ‘No Personal Motive?’ Volunteers, Biodiversity, and the False Dichotomies of Participation. Ethics, Place & Environment, 9(3), 279-298. doi:10.1080/13668790600893319Mao, J., & Song, W. (2008). Empirical study of distinct features and challenges of joint development of information systems: The case of ABC bank. Tsinghua Science and Technology, 13(3), 414-419. doi:10.1016/s1007-0214(08)70066-xMenzel, S., Nordström, E.-M., Buchecker, M., Marques, A., Saarikoski, H., & Kangas, A. (2012). Decision support systems in forest management: requirements from a participatory planning perspective. European Journal of Forest Research, 131(5), 1367-1379. doi:10.1007/s10342-012-0604-yMoote, M. A., Mcclaran, M. P., & Chickering, D. K. (1997). RESEARCH: Theory in Practice: Applying Participatory Democracy Theory to Public Land Planning. Environmental Management, 21(6), 877-889. doi:10.1007/s002679900074Peleg, M., Shachak, A., Wang, D., & Karnieli, E. (2009). Using multi-perspective methodologies to study users’ interactions with the prototype front end of a guideline-based decision support system for diabetic foot care. International Journal of Medical Informatics, 78(7), 482-493. doi:10.1016/j.ijmedinf.2009.02.008Pretty, J. N. (1995). Participatory learning for sustainable agriculture. World Development, 23(8), 1247-1263. doi:10.1016/0305-750x(95)00046-fReed MS. 2008. Stakeholder participation for environmental management: a literature review. Sustainability Research Institute, School of Earth and Environment, University of Leeds.Reed, M. S., & Dougill, A. J. (2010). Linking degradation assessment to sustainable land management: A decision support system for Kalahari pastoralists. Journal of Arid Environments, 74(1), 149-155. doi:10.1016/j.jaridenv.2009.06.016Rowe, G., & Frewer, L. J. (2000). Public Participation Methods: A Framework for Evaluation. Science, Technology, & Human Values, 25(1), 3-29. doi:10.1177/016224390002500101Schielen, R. M. J., & Gijsbers, P. J. A. (2003). DSS-large rivers: developing a DSS under changing societal requirements. Physics and Chemistry of the Earth, Parts A/B/C, 28(14-15), 635-645. doi:10.1016/s1474-7065(03)00109-8Sheppard, S. R. J., & Meitner, M. (2005). Using multi-criteria analysis and visualisation for sustainable forest management planning with stakeholder groups. Forest Ecology and Management, 207(1-2), 171-187. doi:10.1016/j.foreco.2004.10.032Thursky, K. A., & Mahemoff, M. (2007). User-centered design techniques for a computerised antibiotic decision support system in an intensive care unit. International Journal of Medical Informatics, 76(10), 760-768. doi:10.1016/j.ijmedinf.2006.07.011Webler, S. T., Thomas. (1999). Voices from the Forest: What Participants Expect of a Public Participation Process. Society & Natural Resources, 12(5), 437-453. doi:10.1080/089419299279524Van Meensel, J., Lauwers, L., Kempen, I., Dessein, J., & Van Huylenbroeck, G. (2012). Effect of a participatory approach on the successful development of agricultural decision support systems: The case of Pigs2win. Decision Support Systems, 54(1), 164-172. doi:10.1016/j.dss.2012.05.002Von Geibler, J., Kristof, K., & Bienge, K. (2010). Sustainability assessment of entire forest value chains: Integrating stakeholder perspectives and indicators in decision support tools. Ecological Modelling, 221(18), 2206-2214. doi:10.1016/j.ecolmodel.2010.03.02